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Evaluacion y responsabilidad en la preparacion universitaria y profesional: Desarrollando un 
nuevo paradigma. 

Resumen: A medida que las escuelas de todo el pais se preparan para nuevos estandares en el marco 
del Common Core (Nucleo Basico de contenidos), los estados estan creando sistemas mas alineados de 
evaluacion y responsabilidad. Este documento recomienda un enfoque de evaluacion y 
responsabilidad que se centra en el aprendizaje significativo, habilitado por educadores profesionales 
cualificados y comprometidos, y con recursos adecuados y apropiados, de manera que todos los 
estudiantes independientemente de su origen esten preparados tanto para completar estudios 
universitarios y carreras profesionales cuando se graduen de la escuela secundaria. Sobre la base de 
las practicas ya establecidas en otros estados y en las opiniones de responsables politicos y expertos 
en educacion, este documento propone principios para los sistemas de evaluacion y responsabilidad 
eficaces y se imagina lo que un nuevo sistema de evaluacion y responsabilidad podrfa parecer en un 
imaginario ’’Estado 51" en los Estados Unidos. Si bien sera necesario tener una gran discusion y 
debate antes de que un nuevo enfoque puede tomar forma, el objetivo de este trabajo es que 
comenzar la conversacion por lo que el pais pueda cumplir con sus aspiraciones para preparar a los 
estudiantes para que los estudiantes esten preparados tanto para la universidad y carreras 
profesionales. 

Palabras clave: evaluacion y responsabilidad; multiples medidas; aprendizaje significativo; 
habilidades de aprendizaje mas profundas; recursos rendicion de cuentas; capacidad y 
responsabilidad profesional; Estandares Estatales Comunes; evaluaciones basadas en el rendimiento; 
universidad; carreras profesionales. 

Avalia§ao e responsabiliza§ao na prepara§ao universitaria e professional: Desenvolvendo um 
novo paradigma. 

Resumo: As escolas de todo o pais se preparam para novos padroes no ambito do Common Core 
(Nucleo Comum de conteudos), e os estados estao criando sistemas de presta^ao de contas avalia^ao 
mais alinhados. Este documento recomenda uma abordagem para avalia^ao e presta^ao de contas 
que se concentra nas aprendizagem significativa, ensinados por profissionais qualificados e 
educadores comprometidos e recursos adequados e apropriados para que todos os alunos, 
independentemente da sua origem, sejam bem preparados para concluir as faculdade e desenvolver 
carreiras quando se formarem no ensino medio. Com base em praticas ja estabelecidas em outros 
estados e nas visoes de politicos e especialistas em educa^ao, este trabalho propoe principios para 
desenhar sistemas de avalia^ao e responsabilidade efetiva e imagina o que um novo sistema de 
avalia^ao e presta^ao de contas que poderiam aparecer em um imaginario "Estado 51" nos Estados 
Unidos. Embora vai ser necessario ter uma grande discussao e debate antes pode estabelecer uma 
nova abordagem, o objetivo deste trabalho e iniciar a discussao para que o pais possa cumprir as suas 
aspira^oes para preparar os alunos tanto para os estudos universitarios e carreiras profissionais. 
Palavras-chave: avalia^ao e presta^ao de contas; medi^oes; aprendizagem significativa; habilidades 
de aprendizagem mais profundas; presta^ao de contas; capacidade e responsabilidade profissional; 
Normas estaduais do nucleo comum; avalia^oes baseadas em desempenho; Universidade; carreiras. 
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Introduction 

As new college- and career-ready standards for learning are being adopted by virtually every 
state across the country, it has grown clear that many states and communities see the need to move 
toward more aligned systems of assessment and accountability that support genuinely higher and 
deeper levels of learning for all students, and more flexible designs for schools so that their graduates 
can meet the challenges of a world in which both knowledge and tools for learning are changing 
rapidly. 


Outline of the Paper 

This paper outlines a proposal for a new approach to accountability that is responsive to 
these demands, drawing on the experiences of states and nations that have tackled these challenges, 
as well as research that has evaluated the consequences of different approaches to educational 
improvement. 1 It focuses primarily on how states might construct well-aligned systems for assuring 
high-quality education for all students, and treats aspects of the federal role and local activities from 
that perspective. 

In the first section, we set out some principles for effective accountability systems. In the 
second section, we imagine how these principles might be enacted in an imaginary “51st state,” as an 
illustration of one of the many ways the principles might be applied. We were advised and assisted in 
this process by a group of individuals deeply knowledgeable about policy and school improvement, 
who had convened to tackle the question of what a new accountability system might look like. 2 In 
the final section, we present examples of how elements of these proposals are already being enacted 
in some states and communities, in order to offer concrete form to some of the ideas. 

Background 

Policymakers and practitioners have learned a great deal from the experiences of the last 25 
years and can build on educational improvements accomplished under both Democratic and 
Republican administrations. Our next steps should preserve the positive gains achieved as a result of 
a collective commitment to all of our children, while responding to current realities and concerns. 
Under the Improving America’s Schools Act during the Clinton administration, we began the 
process of organizing school improvement around standards for learning, and measuring those 
standards periodically with state assessments, which included, in many states, portfolios and 
performance tasks assessing higher-order skills. Under No Child Left Behind (NCLB) during the 
Bush administration, we articulated a commitment to pursuing higher and more equitable outcomes 


1 Much of this research is summarized in Darling-Hammond, L. (2010). The flat world and education: How 
America’s commitment to equity will determine ourfuture. New York: Teachers College Press. 

2 This paper was developed in the course of a series of discussions about the design of a new accountability 
system, convened by the Hewlett Foundation. The participants in these discussions offered substantial input 
and ideas. There was strong agreement about many of the ideas, and there were diverse perspectives about 
some. The final product reflects many of the individual and collective insights of the participants, but it does 
not reflect an endorsement by any of these individuals or the organizations with which they are affiliated. 
These intellectual contributors include, in addition to the authors: Stephen Bowen, Anthony Bryk, Richard 
Carranza, Michael Cohen, Michael Kirst, Paul Leather, Philip Lovell, Carmel Martin, Jal Mehta, Charmaine 
Mercer, Rick Miller, Chris Minnich, Scott Palmer, Arun Ramanathan, and Larry Rosenstock. 
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for children across social groups, and a commitment to providing well-qualified teachers for all 
children. 

Since 2002, these efforts have been pursued largely through test-based accountability 
strategies that have articulated annual targets for growth, along with consequences for not meeting 
those targets. Noticeable gains have been registered on the state tests that have been the focus of 
these accountability efforts. However, progress has been less evident on the National Assessment of 
Educational Progress (NAEP), where 8th- and 12th-grade scores have been largely flat. And on the 
Program for International Student Assessment (PISA)—a more open-ended test evaluating how 
students can apply their knowledge and can demonstrate their reasoning—U.S. performance has 
declined in math, reading, and science between 2000 and 2012, both absolutely and in relation to 
other countries. On all of these measures, large and persistent achievement gaps remain among 
students by income, language background, and racial and ethnic group. 

It is clear that the NCLB legacy that “every child matters” represents an evolution in our 
thinking. It is also clear that our current strategies are not sufficient to ensure that, indeed, every 
child will be enabled to learn the higher-order skills that they need to acquire to succeed in today’s 
world. The fuller array of deeper learning outcomes students need to acquire include the knowledge, 
skills, and dispositions needed to foster critical and creative thinking, problem solving, collaboration, 
multiple modes of communication, uses of new technologies, the capacity to learn to learn, and the 
social-emotional intelligence that fosters a growth mindset and supports resilience and 
resourcefulness. The broadened definitions of readiness being adopted by states, along with 
proposals emerging under recent ESEA flexibility waivers, are creating demand for greater 
investments in rich curriculum; sophisticated teaching; and new, more robust assessment systems 
that go beyond the multiple choice approaches that have been prominent since 2001. 

The emerging paradigm for accountability must be anchored in this new vision for learning 
and should be coherently aligned to systemic changes implied by that goal. It should foster a culture 
of inquiry and continuous improvement at all levels of the system. This new accountability model 
must foster collaborative change that can transform schools from the industrial model of the past to 
innovative learning systems for the future. Accountability will need to build school capacity and 
enable thoughtful risk-taking informed by continuous evaluation to inform improvement. 

While it is evident that we must pursue new assessment and accountability systems, we 
should learn from the accumulated wisdom of recent experiences. We know that supporting student 
growth is as important as tracking the status of a child’s achievement. We know it is important to pay 
constant attention to children’s progress, and we must maintain systems for determining how student 
learning is advancing each year. We must work toward a clear vision of what proficiency means for 
student performance, anchored in realistic and defensible standards. We must hold ourselves 
accountable for the success of all groups of students. We must develop more informative reporting 
systems and be more transparent in our communication with parents. Our evolving standards must 
accommodate a broad set of knowledge, skills, and aptitudes. And, our new designs must allow us to 
compare student learning within and across schools and districts. 

Additionally, we must be prepared to challenge ourselves to take the next steps to ensure we 
are on track to developing systems to support success for all learners. We are positioned to move to 
a system of multiple assessments “of, for, and as learning,” with curriculum-embedded local 
performance assessments embodying and supporting learning in classrooms, along with richer and 
more meaningful assessments that evaluate learning at the state and local levels. 

We propose this new approach knowing that it is an intermediate step forward that is 
designed within the constraints of the current educational system. We realize that the experience and 
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hard work of practitioners has expanded our vision of what is possible and our knowledge of how to 
implement this new vision. We will know a lot more because of innovations in policy, research, and 
practice that are challenging prior assumptions about what is taught, how students learn, when 
learning occurs, and where learning happens. It is our desire that this design supports those who are 
creating more personalized learning anchored in deeper learning, competency-based learning, and 
student agency. It is our hope that this next-best-step-forward we are proposing will be evaluated, 
improved, and enhanced as the work evolves. No system should be frozen for extended periods of 
time to the point where we find ourselves now: in a place where the system inhibits our ability to do 
what we learn is best for the students we serve. 

A New Approach to Accountability for Learning 

Genuine accountability must both raise the bar of expectations for learning—for children, 
adults, and the system as a whole—and trigger the intelligent investments and change strategies that 
make it possible to achieve these expectations. It must involve communities, along with professional 
educators and governments, in establishing goals and contributing to their attainment. It must attend 
to parents 5 desires and students 5 rights to be taught relevant skills that will matter for their future 
success by competent and caring professionals in adequately resourced schools that are responsive to 
their needs. 

Such genuine accountability will nurture the intrinsic motivation needed to develop 
responsibility on the part of each actor at each level of the system. Thus, a new paradigm for 
accountability should rest on three pillars: a focus on meaningful learning, enabled by professionally 
skilled and committed educators, supported by adequate and appropriate resources. 

It should be animated by processes for continuous evaluation and improvement that lead to 
problem solving and corrective action at the local level, supported by the state. 


Meaningful Learning 



Figure 1 . Key elements of an accountability system 

Such a system should be: reciprocal and comprehensive, focused on capacity-building, 
performance-based, and embedded in a multiple-measures system. A reciprocal approach means, 
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first of all, that each level of the system should be held accountable for the contributions it must 
make to produce an effective system. A comprehensive system must attend to the inputs, processes, 
and outcomes that produce student learning: In others words, it must build capacity to offer high- 
quality education, while holding educators accountable for providing such education. In addition to 
adequate, intelligently allocated resources and professional expertise, this should include developing 
problem-solving capacity that guides ongoing improvement, informed by data and by processes such as 
strategic planning, evaluation, and school quality reviews that identify and correct problems in 
effective ways. Intelligent evaluation of accomplishments, needs, and next steps that can guide 
diagnosis and improvement requires a dashboard of useful measures of student, educator, school, 
and system efforts and outcomes that are developed at both the state and local levels. 

Accountability for Meaningful Learning 


If meaningful learning for all students is the focus of an accountability system, the system 
should use a range of measures that encourage and reflect such learning, and it should use those 
measures in ways that improve, rather than limit, educational opportunities for students. This means 
we need both much better assessments of learning—representing much more authentically the skills 
and abilities we want students to develop—and multiple measures of how students, educators, 
schools, districts, and states are performing. 

These skills and abilities include both the applications of content knowledge reflected in new 
learning standards and the “soft skills” that allow people to be strategic in their learning. For 
example, David Conley’s description of skills needed for college and career readiness includes key 
cognitive strategies , such as problem formulation, research, interpretation, communication, precision, 
and accuracy; key content knowledge , including the structure of knowledge; key learning skills and 
techniques that allow learners to be conscious of how they learn and capable of taking ownership of 
their learning; and key transition knowledge and skills that allow young people to understand and manage 
the context, processes, cultural and personal factors, and financial dimensions of the decisions they 
might make as they move into college and career settings (Conley, 2013). 


FOUR KEYS TO COLLEGE AND CAREER READINESS 
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Figure 2. Keys to college and career readiness 
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A system of higher-quality assessments , both state-designed and locally developed, should include 
authentic performance tasks (e.g., classroom-based projects and products like those used in other 
countries) that assess and encourage the development of the full range of higher order skills. These 
kinds of assessments should be part of student learning evaluations and should also be part of a 
multifaceted collection of evidence for teacher evaluation and school review. Moving to a system of 
assessments necessitates that we abandon a singular focus on statewide summative assessments as the 
basis of all important decisions. 

As the CCSSO Accountability Advisory Committee (2014) recommended: 

Each state should establish rigorous statewide measures of CCR (such as through 
Common Core-aligned assessments), but should also provide latitude for district 
innovation to expand on those measures to include additional indicators of CCR 
skills or dispositions deemed important by the local community. 

As in jurisdictions like Australia, Finland, and Singapore, the standardized measures can be used to 
validate the local assessment results, while the performance assessments are used to inform 
instruction, provide feedback to students and teachers, and enable diagnostic decisions, as well as to 
provide evidence of student learning. Both should be part of a research and development process to 
validate the assessments and to provide evidence of their effects on instruction and learning. 

As performance tasks offer more detailed information about how students think and 
perform, they are more useful for formative purposes, although they can offer information for 
summative judgments as well. Many school districts are routinely using digital tools that engage 
students in embedded performance assessments as an inseparable part of the learning process. In a 
new system of assessments, it should be possible to move from an overemphasis on external 
summative tests, even as they become better representations of what students should know and be 
able to do, to a greater emphasis on assessment that can shape and inform learning. This strategy will 
reduce the “overtesting” burden, shifting time and energy from external summative events to 
formative assessments that can be used in more efficient and effective ways. (See Figure 3 below.) To 
achieve these benefits, we will need to rely more on adjudication at the local level where learning 
occurs. This implies more trust of professionals who are highly trained and supported with judgment 
tools and processes, such as common rubrics along with moderation and auditing processes for 
evaluating student work consistently. 

Formative Interim Summative 


Interim Summative 


Figure 3. Relative emphasis on assessment purposes (P. Leather, personal communication, September 
3,2013) 

In a new system of accountability, multiple measures , coupled with thoughtful systems of 
judgment, should be used to inform decision making at each level. Transparency in providing 
information to the public and to educators and policymakers is a key aspect of the new 
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accountability. Like businesses that use a dashboard of measures to provide a comprehensive picture 
of performance, we need a dashboard of indicators to inform key decisions (student placement and 
graduation; teacher evaluation, tenure, and dismissal; school recognition and intervention). Full and 
timely reporting of a wide array of information to parents and the community is a basic element of 
accountability. In line with professional standards, test scores should never be used alone for any 
such decision. Data should be thoughtfully interpreted and weighed by experts who make decisions 
based on multiple sources of evidence. 

Through the federal waiver process for ESEA flexibility, states have already begun to 
incorporate broader measures into their accountability systems. Ultimately, long-term outcomes, 
such as success in negotiating college and careers, can become the true accountability measures. In 
the immediate future, a number of leading indicators can become part of state accountability 
systems. When evaluating schools, multiple measures of student learning can be coupled with other 
indicators of important education outcomes, such as, 

• students 5 social-emotional competence, responsibility, citizenship, etc.; 

• teachers’ professional contributions to the professional team and the school as a whole, 
as well as evidence of individual practice; and 

• school graduation rates, attendance, evidence of school climate (through surveys of 
teachers, students, and parents), rich curriculum opportunities, indicators of college and 
career readiness, and measures of successful transition to postsecondary learning and 
work. 

This information should be used in a system that makes strategic investments in educational improvement 
rather than being used mechanically to mete out sanctions. 

Resource Accountability in a Reciprocal System 

Accountability tools must address the barriers to good education that exist not only within 
schools and classrooms, but at the district, state, and national levels as well. For although schools 
themselves may be appropriately viewed as a key unit of change in education reform, the structuring of 
inequality in learning opportunities occurs outside the school in the governmental units where funding 
formulas, resource allocations, and other educational policies are forged. In sum, if students are to be 
well-served, accountability must be reciprocal. That is, federal, state, and local education agencies must 
themselves meet certain standards of delivery while school-based educators and students are expected to 
meet certain standards of practice and learning. 

Thus, in addition to learning standards that rely on many kinds of data, accountability must 
encompass resource standards . With the advent of more challenging and authentic measures of student 
performance, the creation of accountable schools and school systems will demand methods for inspiring 
and ensuring equitable access to necessary learning opportunities, so that all students can achieve these 
learning goals. This means that local decisions about how people, funds, and time are allocated should 
not be separated from decisions about how the school is performing in relation to student learning. It 
also means that states should design funding policy to address equity and adequacy. 

A complete view of accountability must take into account smarter resource allocation 
throughout the system, including the appropriate roles of states and school districts in supporting local 
schools in their efforts to manage resources more effectively to meet standards. This includes: 

• allocating adequate school resources in relation to students’ learning needs; 

• ensuring equitable access to high-quality curriculum and instructional materials that support 
students in learning the standards; and 
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• providing well-prepared teachers and otherprofessional staff to all students in settings that allow 
them to attend effectively to student needs. 

Professional Capacity and Accountability 

Also critical are professional standards of practice that should guide how educators are prepared and 
how they teach and support students. Accountability for implementing professional practice rests not 
only with individual educators, but also with schools, districts, and state agencies that recruit, train, hire, 
assign, support, and evaluate staff. Collectively, they hold responsibility for ensuring that the best 
available knowledge about curriculum, teaching, assessment, and student support will be acquired and 
used. Individuals and organizations should be responsible for building their own capacity for 
professional practice; they should be accountable for evaluating practice and student progress, and 
engaging in continual improvement based on the results. 

These core building blocks of state accountability systems provide the foundation for schools 5 
capacity to serve their students well: 

• Educator capacity that enables teachers to teach for deeper learning and administrators to 
understand and support this work at the school and district level. Ensuring this capacity 
requires: 

—-high-quality preparation, induction, and professional development; 

—accreditation and licensing based on evidence of teacher and administrator 
performance in supporting diverse learners to meet challenging standards; and 
—evaluation based on multiple indicators of practice, contributions to student 
learning, and contributions to colleagues that supports ongoing learning. 

• School capacity to meet student needs is based on school, district, and state actions that 
ensure: 

—the availability of an appropriate mix of well-qualified staff who are properly assigned and 
adequately supported with professional development, and 

—well-designed curricula and educational programs that are consistent with research. 

• System capacity for professional practice and improvement must be supported by: 

—awareness of research, as well as 

—inspection or school-quality review processes that evaluate policies, programs, practices, 
and outcomes; diagnose areas for improvement; and guide appropriate interventions. 
Professional capacity and accountability are reinforced by a system that has developed professional 
judgment as a key expectation for evaluating the work of students, the work of other teachers, and 
the work of schools. Expert professional judgment, used to make sense of qualitative and 
quantitative information, can support more defensible decisions. In addition, it can help 
professionalize education by serving as a form of professional development for educators, and it can 
support a more genuine sense of responsibility as educators, who work with students and families, 
feel a sense of engaging in accountability themselves, rather than having it imposed externally. 

Finally, a more relational accountability is developed when educators act in a professional community 
with each other and when they interact in learning communities with families—something that can 
prove much more powerful than a more impersonal institutional accountability. 

Continuous Improvement and Corrective Action 

These three elements of a new system—supports for meaningful learning, accountability for 
resources, and accountability for professional practice—provide the grist for specific improvement 
processes that are informed by rich sources of data and diagnostic information about what is 
happening and what is needed to sustain growth and learning, as well as to solve pressing problems. 
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These processes, like quality reviews for schools, use data in combination with expert judgment to 
evaluate progress in ways that provide actionable guidance for improvement. 

They should be accompanied, as needed, by resources that can be directly applied to a 
turnaround effort—for example, the time and skills of expert educators who are trained and funded 
to work with struggling schools in teams, school pairs, or networks; curriculum specialists who can 
help overhaul instructional plans and coach teachers; the availability of wraparound services where 
those are needed to support student welfare and success; models and supports for successful 
afterschool or summer programs; and so on. 

The same general principles should inform thoughtful evaluations for educators, coupled 
with supports for improvement and learning reviews for students. 

New Accountability in the “51st State” 

What might this new accountability model look like in a state that decided to develop all of 
these components in an integrated system? Figure 4 illustrates what the components of the system 
might look like. This is, of course, only one approach among many that could be used to put these 
principles into action. 

Accountability for Meaningful Learning 

The 51st state wants students 5 and teachers 5 work to be focused on the kinds of knowledge 
and skills that will contribute to student success after graduation, developed in relevant and engaging 
ways. The state pursues meaningful learning by: 

1) establishing college- and career-ready standards anchored in core academic knowledge 
and skills that recognize competencies considered by higher education, employers, and 
parents as critical to success; 

2) supporting the development and distribution of high-quality curriculum materials and 
assessment tools for use by teachers and students; and 

3) encouraging local districts to select and develop thoughtful, curriculum-embedded 
assessments of students 5 knowledge and skills that provide ongoing diagnostic 
information to support learning. 

The state also plays a role in validating district and school outcomes and intervening in 
underperforming districts and schools to support corrective action. 
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The system is premised on multiple measures, which include, as one component, robust 
local assessments that can evaluate deeper learning skills, as well as state standardized assessments 
of student performance to verify the results of local assessments. Such state validation could occur 
every year for every child, or at points in the grade spans that represent critical developmental 
junctures (for example, grades 3 or 4, 7 or 8, and 11 or 12), or differentially, depending on local 
needs. State assessments employ matrix sampling so that judgments can be made about a broader 
and deeper set of skills without overtesting children. Disaggregation of results is part of the reporting 
system for assessments. 

Annual determinations of progress are maintained for every child at the school and district levels. 
These determinations are made more meaningful through tools that assess student movement along 
learning progressions (e.g., the Developmental Reading Assessment, the STEP reading assessment, 
writing portfolios providing evidence of growth in multiple genres along a continuum reflected in 
shared rubrics, and assessments of progress in mathematical thinking and skills along key 
progressions). Most local assessments are designed to be embedded in the curriculum, just as 
teachers 5 assessments in the form of papers, projects, presentations, quizzes, and other diagnostic 
evaluations currently are. However, these are designed to provide much richer diagnostic 
information more aligned to the new standards than many local assessments currently offer. 

The 51st state recognizes that students learn in different ways at different rates so that 
growth is benchmarked against learning progressions rather than grade levels. It also recognizes that 
students may progress at different rates in different disciplines or skill areas, and students are served 
much more flexibly than in our current fixed organizational structures. Districts can use state- 
developed or approved tools to track student progress (including common tasks assembled in an 
assessment bank, for example), or they can develop their own and bring them to the state quality 
assurance panel for approval. 

State validations of student learning include assessments in English language arts, 
mathematics, and science that combine sit-down tests with structured performance tasks (e.g., 
writing samples taken individually or organized in structured portfolio collections, mathematics 
applications, and structured scientific investigations). Locally administered tasks allow students to 
develop and demonstrate complex college- and career-ready skills that require more time and 
different modes of demonstration than a short sit-down test can accommodate: inquiry skills, 
written and spoken communication, ability to use feedback to revise, uses of technology, etc. The 
state provides common rubrics, training for scoring, and auditing to ensure that these can be scored 
reliably. Teachers are involved in designing and scoring open-ended items and tasks in both the state 
and local assessments as a means for professional learning about the standards as well as for sharing 
strategies for designing curriculum and teaching to meet the standards. 

Together, these comprise a system of assessments using both state and local sources of 
information: standardized test measures of certain aspects of students 5 learning that are assessable in 
a testing context—including performance elements that measure some higher-order analytic skills. 
These are augmented by more robust local performance assessments that can support and evaluate 
harder-to-measure abilities: the ability to design and conduct extended investigations, to collaborate, 
to communicate in multiple forms, to persevere, to exhibit resilience, to use feedback productively, 
and learn to learn. 
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Figure 5. Elements of the assessment system 

Measures embedded in local assessment programs that are used for state accountability 
purposes may be approved through Assessment Quality Assurance Processes (which can take the form of 
a panel comprised of expert practitioners and other curriculum and assessment experts, or other 
approaches to peer review). These processes are designed to ensure that the assessments and the 
ways they are applied (rubrics, scoring procedures, uses of results) are appropriate (e.g., that they 
measure the standards well and with high fidelity, are valid and can be reliably scored, and are used 
appropriately). 

At both the state and local levels, curriculum and assessments support and reflect deeper 
learning skills , including critical thinking, problem solving, communication, collaboration, creativity, 
and the ability to learn to learn. The system also supports the development of social-emotional skills 
that colleges and employers recognize as important and that have both intra- and interpersonal 
dimensions, such as collaboration, resilience, perseverance, and an academic growth mindset, by 
including complex extended tasks that require students to learn how to work with others, to take 
and use feedback productively, to solve problems resourcefully, and to persevere in the face of 
ambiguity and problems. These kinds of tasks are, necessarily, embedded in the local curriculum, but 
those used for student or school judgments are scored with common rubrics , using moderation and 
auditing processes to achieve consistency when they are used as part of the reporting for accountability 
purposes. 

State assessments address some of the key deeper learning skills as well, in less extended 
tasks, so as to signal what is valued and attended to. Local assessments can go further to foster and 
assess student initiative and choice, calling on students to be agents in their own learning by 
requiring them to design and complete their own investigations, assemble evidence about their 
progress and skills, and orchestrate collaborations that lead to the creation of products (e.g., 
software solutions, engineering designs, data collection and analysis, literary anthologies, topological 
maps, artistic productions, and museum exhibits) that emulate work or are created as a result of 
work in the world outside of school. 
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Table 1 


51st State k System of Assessments 


Types of assessment 

Pre-secondary level 

Secondary level 

Curriculum 

guidance 

Curriculum Resources for New Standards: 

Curriculum frameworks that include 
unit templates, formative 
instructional tools, and performance 
assessment options with quality 
descriptors (rubrics) 

Courses of study with embedded assessments 
(e.g., IB, AP, Linked Learning (CTE), or 

Early College/dual credit pathways, optional 
state courses of study with syllabi, locally 
designed alternatives 

External 

tests 

State assessments validating 
mathematics, ELA, and science 
learning at each grade span, one test 
per grade in grades 3-5, 6-8, and 9-11 
(subjects may alternate at different 
grade levels—see note below) 

Consortium College and Career Ready Test, 
at grade 11 or when ready, including 
re search/writing task and mathematical 
application 

Common 
performance 
tasks, locally 
administered 

Common Assessment Tasks: 

Common performance tasks 
evaluating inquiry in science and 
social studies once per grade span; 
guidance for arts, writing, and 
technology tasks or portfolios 

Common assessments embedded in courses 
of study; guidance for exhibitions of mastery 
in different fields, including competency- 
based badging or micro-credentialing 

Locally 

developed 

assessments 

Local performance assessment 
systems—locally scored and 
internally moderated 

Graduation portfolios supporting student 
profiles, guided by state standards—locally 
scored/externally moderated 


Note: Although this description references classrooms, courses, and grade levels, the 51st state is moving toward a 
competency-based approach to education, which allows students to be assessed along a broader continuum of learning 
and achievement, using specific tests or tasks when they are appropriate for the individual student without regard to age 
or grade level. 

At the capstone level, in addition to the Consortium assessment of college- and career-readiness at 
grade 11, students develop and maintain a porfolio of evidence (drawn from the assessments already 
described) regarding their performance in key areas of the curriculum and a profile of their 
accomplishments that can be communicated to colleges and employers. The portfolio serves as 
evidence that the student has met core competencies for readiness and has also prepared to meet 
personal goals for next levels of learning and work. Students complete some components in 
common and complete others that illustrate their unique talents and specialized studies and skills in 
chosen pathways. The common components are used to demonstrate college- and career-ready 
competencies that have been shown to be associated with postsecondary success: 

• research and inquiry skills that require critical thinking and analysis (generally 
demonstrated in scientific investigations or social science research); 

• quantitative reasoning applied to a real-world problem (through the use of statistical 
analysis in the science or social science investigations above, for example, or a project 
designed to illustrate mathematical problem-solving); 

• communication skills (written and oral); 

• collaboration skills; and 

• use of technology for investigation and presentation of information. 
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These may be illustrated through tasks that are constructed to illustrate the mastery of disciplinary 
modes of inquiry in fields like science or history, or tasks that engage students in interdisciplinary 
problem solving. The competencies are incorporated into common rubrics; tasks are scored with 
moderation. Students are also encouraged to include demonstrations of competence in other areas, 
for example: 

• world language—a demonstration of proficient communication in a language other than 
English, through a recorded conversation or a written paper or letter; 

• arts—a demonstration of performance in an area of the performing arts; and 

• career/technical education—a demonstration of competence outlined in a career 
pathway (often developed with industry). 

These components should be completed as part of the assessments already planned in a school, 
refined to meet a “portfolio standard ,,” and may be drawn directly from a student’s participation in an 
existing program of study, such as the International Baccalaureate program or a College Board suite 
of courses that include such assessments. Schools that participate in the New York Performance 
Standards Consortium, many Linked Learning schools, and schools in Deeper Learning networks 
will also have already developed portfolios that address these expectations. The state provides a set 
of models for districts to use if they so desire. At least one of these components should be defended 
before a panel that allows students to share and explain their work orally and in writing with a panel 
of teachers, other students, and community members, and to respond to questions. 

This compilation of evidence is assembled with other evidence about students’ 
accomplishments (e.g., grades, test scores, extracurricular activities, work experiences, letters from 
employers or teachers) and a reflective statement from students about their experiences and goals in 
a student profile that can be used as a tool to guide student advisement, goal-setting, and 
communication with colleges and employers. 

The state has developed a platform in which students can upload the profile and their work 
samples into a digitalportfolio that can be used by employers and postsecondary institutions for 
admissions, advisement, and placement. The portfolio includes a summary that makes key evidence 
easily understood by a user within 10-15 minutes—providing summary data, a short writing sample, 
a short videotape of the student presenting a learning demonstration, and a table of contents that 
can direct those who want more information to a link. Some users will look only at the summary 
data. But a college considering a student for an art major could look more deeply at the art portfolio, 
while an employer wondering about a student’s oral skills and career and technical knowledge could 
click on the link to the presentation about a design solution that the student developed. Students 
carry their portfolio with them after high school to support their strategies for postsecondary 


success. 
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Summary: transcript, GPA, CCR test scores, statement of goals, distinctive accomplishments or 
"badges," short essay, 2-minute video clip from portfolio presentation, table of contents 




Investigation of climate change trends in a local community (science 
and mathematics), includes paper, data set, and PowerPoint 


& 


What social and political forces influenced the passage of the 14th 
Amendment to the Constitution? (historical inquiry) 




The American Dream in 20th century literature (literary analysis), 
includes videotaped presentation to panel 


J, 


Demonstration of competence in world language : Tamil 
(audiotaped conversation and paper) 


Figure 6. Digital portfolio 

Accountability for Adequate and Intelligently Used Resources 

The 51st state has pursued resource accountability by developing a weighted student funding 
formula that allocates funds based on pupil needs, allocating a greater weight to students living in 
poverty, English learners, and students in foster care. By providing resources more equitably, the 
state can expect schools that serve high-need students to provide the wraparound services that will 
enable children to come to school healthy and ready to learn and can ensure that they are adequately 
supported once they are there. In addition, the state holds districts accountable for intelligent and 
equitable use of funds by requiring that local communities be involved in decision making about 
budgets and programs, and by tracking key inputs and results for all districts and schools. 

Transparency is a key aspect of the accountability strategy. A multiple measures system of 
accountability includes a dashboard of indicators—some required by the state for all schools and 
others proposed and tracked by local communities that have a voice in the accountability process. 
The measures include evidence about both outcomes and inputs, supporting diagnosis of what is 
working and what is not. Like the dashboard on a car, which provides indicators of speed, distance 
traveled, fuel, fluids, tire pressure, and more, the combination of measures signals where to look 
further to figure out how things are working. Outcome data are disaggregated by student race and 
ethnicity, poverty, language status, and disability status. 

The report card for each school indicates current status and progress on each of the 
measures, much like the reporting system used in Alberta, Canada. (See Appendix A.) Thus, the 
public has access to evidence provided by districts and schools about what they offer their students 
and what the outcomes are; schools can see where they are doing well and where they may focus 
improvement efforts, and the state has a well-organized set of indicators about how schools are 
progressing and which ones need further assistance. 
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Dashboard of Multiple Measures: Outcomes 


State & Local 

Assessments 

Consortium tests 

Performance assessments 

English language proficiency gains 

Assessments of college & career- 

ready skills: AP, IB, CTE 

Instructional expenditures 
Educator qualifications 


Student Participation 

Attendance 
Persistence rates 
Graduation (4, 5, & 6 year) 
Expulsion/Suspension 
Postsecondary transition 
2 nd year enrollment in IHEs 

Inputs / Context 

Student characteristics 
Student supports 


School Climate / Opportunity to Learn 

Student 

Parent Surveys 

Teacher 

% completing CCR courses of study 
Social-emotional learning & supports 


Curriculum offerings 
Extracurricular opportunities 


Figure 7. Dashboard of multiple measures: outcomes 

Corrective action. These data are the grist for a School Quality Review system that helps 
schools assess their practices and work on areas for improvement, and that supports intervention and 
corrective action in schools where the evidence suggests that achievement is not adequate and students’ 
needs are not being met. 

The School Quality Review process brings together several elements that have not been 
joined before in most education policy systems: robust data^ educational expertise , and peer review. Like the 
Inspectorate model used in many countries abroad, it is guided by experts who are deeply 
knowledgeable about practice and well-trained in how to conduct a diagnostic inquiry into school 
practices and their relationship to the nature and quality of student learning. [Similarly, states like 
Kentucky and North Carolina have formed teams of expert educators (often highly accomplished 
teachers and administrators) to diagnose and help address the needs of low-performing schools.] 

Like U.S. accreditation systems, the engagement of peer reviewers from other schools in the state 
brings multiple perspectives to the task while stimulating a learning process for participants that 
expands their knowledge and sharpens their analytical skills. Like many research endeavors, the 
skillful use of robust quantitative data, much of which is comparable across schools, with qualitative 
insights developed from looking purposefully at teaching and student work and talking to 
stakeholders, allows reviewers to get a better understanding of how the school is working and what 
may help it improve. By combining these things, the process is more powerful and purposeful than 
accreditation approaches have been in the past. 
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Figure 8. School quality review 

In the 51st state, the School Quality Review process is available to all schools on a cyclic 
basis (typically every 5th year), and to schools that volunteer to participate more frequently because 
they want the additional help it can provide. It is activated immediately for schools that are identified 
by red flags associated with their students 5 achievement, participation, or opportunity-to-learn 
outcomes (low performance, little improvement, or large equity gaps). The Review is joined with an 
intensive support process in which the district and state identify and activate the human and other 
resources that are needed to enable the school to turnaround its practices and student performance. 
The system of identification for intervention is based on a set of criteria for school conditions and 
progress, rather than on a norm-referenced percentage of schools. 

A support capacity has been built to work with schools or districts that request or are identified 
for improvement assistance. The support structures include: 

• training and deployment of a cadre of Distinguished Educators—accomplished teachers, 
principals, and superintendents—who are intensively trained and made available to work 
with schools and districts that are engaged in intensive improvement or turnaround 
efforts; 

• support for pairings and networks of schools focused on sharing expertise for the 
purpose of school improvement; 

• professional development for school leaders and school teams implementing new 
curriculum standards, using assessments to inform improvement, and developing school 
improvement initiatives, including more productive professional learning communities 
and Peer Assistance and Review Programs; and 

• training of mentors for teacher and administrator induction and coaches for veteran 
teacher support. 

These structures build the capacity of schools to do their work well, while ensuring that students are 
not left to languish in schools that are performing poorly. 


Accountability for College and Career Readiness: Developing a New Paradigm 


19 


Professional Capacity and Accountability 

Finally, the 51st state works to ensure professional capacity and accountability in a number of 

ways: 

1) It has strengthened initial entry into the profession for teachers and administrators by: 

• strengthening expectations for programs to develop candidates 5 capacities to teach the 
Common Core State Standards and to work with diverse learners (including 
economically disadvantaged students, special education students, and English 
language learners). These capacities include a strong understanding of student 
learning and development; curriculum, instruction, and assessment within the 
content areas to be taught; classroom management; and how to work collaboratively 
with colleagues and parents; 

• sharing information about successful program models; 

• investing in stronger clinical training models through residencies and professional 
development schools; 

• evaluating candidates 5 readiness to teach and lead through teacher and administrator 
performance assessments for licensing and feeding results back into programs for reflection 
and improvement; 

• leveraging higher quality preparation through performance-based accreditation that 
examines program results (through pass rates on teacher and administrator 
performance assessments; graduate and employer surveys, entry and retention rates 
in teaching and administration, and evidence of graduates 5 later effectiveness) as part 
of a more serious accreditation process; 

• supporting high-quality induction by training and supporting the time for mentors to 
work closely with beginning teachers and administrators. 

2) It has built on this stronger foundation to develop professional learning systems that: 

• offer high-quality curriculum resources (including instructional materials and videotapes 
of practice) around which professional development can be organized and on which 
teacher teams can build, try, and refine locally adapted lessons and instructional 
strategies; 

• organize sustained\ high-quality professional learning opportunities for networks of educators 
(e.g., through subject matter projects) focused on developing practice through 
extended institutes, collective inquiry, action research to solve complex problems of 
practice, and coaching; 

• provide incentives for schools to establish flexible structures within the teaching day 
and year that provide time for teachers to participate in collegial planning and job- 
embedded professional learning opportunities; 

• provide ongoing training for schools to develop effective professional learning 
communities that can analyze student learning and school progress in relation to 
practice, and engage in ongoing improvement. 

3) It has helped local districts build stronger evaluation systems that: 

• are based on professional standards that are used to assess educators’ practices from pre¬ 
service preparation to induction and through the remainder of the career; 

• combine evidence from several sources , including standards-based measures of educator 
practice and valid evidence of student learning that is appropriate to the curriculum 
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and students being taught. These are examined in relation to one another, along with 
evidence of professional contributions to school improvement; 

• include opportunities for both formative and summative evaluation, providing 
information both to improve practice and to support personnel decisions; 

• tie evaluation to useful feedback and to professional learning opportunities that are 
relevant to educators 5 goals and needs; 

• acknowledge the time, curriculum resources, and professional learning needed to 
learn to implement more complex standards, such as the CCSS and NGSS; 

• differentiate supportb&sed on the educator’s level of experience and individual needs; 

• build on successful Peer Assistance and Preview models for educators who need assistance 
(both administrators and teachers) to ensure intensive, expert support and well- 
grounded, timely, and effective personnel decisions; 

• value and promote collaboration , which feeds whole school improvement; 

• are a priority within the district, with dedicated time , training, and support provided to 
evaluators and to those who mentor educators needing assistance. 

4) It has promoted equity in the provision of expertise to students by: 

• equalising resources to districts while tracking and encouraging the provision of 
well-qualified and effective teachers to all schools; 

• creating a greater supply of experienced, qualified, in-field, and effective 
teachers to high-need schools through service scholarships to recruit a diverse 
pool of high-ability educators to high-need fields and locations by paying for 
their preparation in exchange for at least 4 years of service in the state’s 
schools and through teacher residency programs that recruit, prepare, and mentor 
candidates to learn to teach well in high-need districts; and 

• building professional capacity through the state by creating a statewide learning system , 
and developing a State Education Agency that sees its job as building professional 
expertise rather than just managing compliance. This agency shares research and best 
practices through its website and dissemination activities (newsletters, conferences, 
school quality review activities); documents and disseminates what is working in schools in 
the state in multiple ways, including case studies, site visits, and tools to support local 
policy and practice; and sets up and supports learning networks that allow districts, 
schools, and educators to learn from one another. 

At the end of the day, policymakers and practitioners hope that these strategies will produce schools 
that are responsible for implementing a strong teaching and learning system and responsive to the 
individual needs of all the students they serve. 

Emerging Elements of a New Accountability 

Many of these elements of a responsible and responsive accountability system are already 
emerging in states and districts across the nation. A few of these are highlighted here. 3 


3 This section draws in part on a report by the Center for American Progress and the Council of Chief State 
School Officers. (2014). Next generation accountability systems: An overview of current state policies and practices. 
Washington, DC: Center for American Progress. 
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Accountability for Meaningful Learning 


About 40 states have been involved in two consortia that are developing new assessments of 
the Common Core State Standards: the Partnership for Assessment of Readiness for College and 
Careers (PARCC) and the Smarter Balanced Assessment Consortium (SBAC). These promise to 
include more open-ended questions and tasks that can better evaluate higher-order thinking and 
performance skills than many state tests included in the past. A number of states are participating in 
an Innovation Lab Network under the aegis of the Council of Chief State School Officers. They are 
strategically designing a variety of ways to develop and assess the full range of Common Core State 
Standards and, beyond those, many of the additional college and career readiness skills—the abilities 
to self-assess, plan, persevere, use feedback, and learn independently—needed for success in the 
world after high school. 



Figure 9. Competencies to be developed and assessed 


New Hampshire, for example, has begun to create a system of state and local performance 
assessments that aims to “promote the use of authentic, inquiry-based instruction, complex thinking, 
and application of learning . . . [and] incentivize the type of instruction and assessment that support 
student learning of rich knowledge and skills.” In addition to the Smarter Balanced Assessments in 
English language arts and mathematics, this system will include a set of state-developed common 
performance tasks in the core academic subjects, plus locally designed assessments made available 
through a web-based bank of local and common performance tasks, and a district peer-review and 
auditing process to ensure validity and reliability. 

Each district will propose to the state a locally designed Performance Assessment of 
Competency Education (PACE) system that will provide measurable outcomes aligned with district 
goals and state priorities. The system will include annual determinations of student achievement and 
growth through locally designed and state-validated systems of performance assessments, and will 
provide external validation of the performance assessments through statewide summative 
assessments of college and career readiness in grades 4 and 8. New Hampshire is supporting 
districts’ development of PACE models by developing common statewide performance tasks and 
the necessary processes, tools, and protocols for validating high-quality tasks aligned to state 
standards. The state is also organizing professional development institutes and regional support 
networks, and is developing a network of practitioner “assessment experts” to support schools. 

The district peer review audit process is intended to help build local capacity to do this work 
well. Peer review teams of external practitioners will review evidence submitted by the district, and 
will also collect additional data and provide feedback according to common criteria during a site visit 
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to the district. According to current designs, the peer review process will be used to provide 
formative feedback to districts during the first 2 years. By the 3rd year, however, the audits will 
become integral to the approval process for districts seeking to implement a Performance 
Assessment of Competency Education model for accountability purposes. 

Kentucky maintained a system of performance assessments for two decades, including a 
writing portfolio and mathematics performance tasks, and is now redesigning its systems around 
Common Core State Standards (evaluated in part through the PARCC assessments) and a college- 
and career-readiness agenda. One element of this new effort has been to free some districts from 
state requirements though legislation creating Districts of Innovation (Dofl). Among these districts, 
Danville has incorporated the portfolio graduation strategies developed by schools in the New York 
Performance Standards Consortium: a set of rigorous, performance-based tasks at the high school 
level that must be presented to a committee, defended, and revised to meet a high standard. The 
tasks include a scientific investigation, a social science research paper, a literary analysis, and a 
mathematical modeling paper, which, when completed at a passing level, waives students in these 
schools out of the New York Regents Exams. Consortium teachers score the tasks in a moderated 
system. Other Kentucky Districts of Innovation are adopting similar strategies. 

House Bill 424 had proposed an amendment to Kentucky’s previously passed Districts of 
Innovation legislation that would allow such districts to apply for modification or waiver of 
provisions of the statewide assessment system if the alternate assessment plan meets the intent of 
the statewide assessment system and is consistent with the requirements of NCLB, its successor, or 
federally granted waiver (Center for American Progress and the Council of Chief State School 
Officers, 2014). 4 Similar to New Hampshire, Kentucky hopes to develop technical guidance and 
capacity to validate locally designed performance-based assessment and accountability models that 
would include external audits via statewide summative assessment in grades 3, 5, and 8. The House 
bill has not yet been passed in the Senate, however. In the meantime, Kentucky is working on plans 
to encourage Districts of Innovation to operate performance-based assessment and accountability 
models while still administering all statewide summative assessments required in statute, at least for a 
transitional period as necessary. 

In Rhode Island, a high school diploma requires successful completion of at least two 
performance-based diploma assessments, the options for which are decided by the district and may 
include graduation portfolios, exhibitions, comprehensive course assessments (50% of which must 
be performance-based and include evaluation of knowledge application), or Certificate of Initial 
Mastery. Districts are charged with developing the performance-based diploma assessments, which 
must include demonstrations of both core content proficiency and applied learning skills, as 
determined by a panel that evaluates the student performance using a state-approved rubric. Within 
the allowed forms of assessment, the Graduation Portfolio option is defined in regulation as a 
“collection of work that documents a student’s academic performance over time and demonstrates 
deep content knowledge and applied learning skills,” with evidence including both required and 
student-selected performance-based demonstrations, reflections, and a final presentation (Center for 
American Progress and the Council of Chief State School Officers, 2014). 5 

Similarly, high school diplomas in Maine are awarded based on demonstrations of 
proficiency around the Maine Learning Results and Guiding Principles (Center for American 
Progress and the Council of Chief State School Officers, 2014), 6 and must take into account, “in 
addition to any local course work and accumulation of credits, a broad spectrum of learning 


4 For further details, go to http://legiscan.com/KY/bill/HB424/2014 

5 For further details, go to http://sos.ri.gov/documents/archives/regdocs/released/pdf/DESE/6433.pdf 

6 For further details, go to http://www.maine.gov/doe/proficiency/about/proficiency-based.html 
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experiences that may include internships, portfolios, long-term capstone projects,” and other 
“appropriate learning experiences that provide opportunities to demonstrate proficiency” (Center 
for American Progress and the Council of Chief State School Officers, 2014). 7 Like New Hampshire 
and Kentucky, Maine is part of Innovation Lab Network activities to build a shared performance 
assessment bank and to use local performance assessments as part of the state accountability system. 

Resource Accountability 

In a reciprocal system, not only does the state hold schools, educators, and students 
accountable for meaningful teaching and learning, but also parents and communities can hold the 
state accountable for allocating resources in a fair and equitable manner and for investing in ways 
that are designed to accomplish the goals of career- and college-readiness. Adequate and intelligently 
used resources thus become part of the accountability system, along with indicators of system 
performance that allow an evaluation of whether appropriate progress is being made at the school 
and district levels. 

California recently adopted a Local Control Funding Formula (LCFF), which brings new 
money into the system that will increase annually over the next 6 years, and allocates all of the 
funding based on pupil needs. LCFF eliminates categorical funding while providing a base grant for 
each LEA based on per average daily attendance, with an extra 20% boost for each disadvantaged 
student (low-income, English learner, or foster care child) and additional funding for those who 
attend schools where at least 55% of students are disadvantaged (California Department of 
Education, 2014). This will reverse the effects of a system that previously provided the least 
resources to the highest-need students. 

The Local Control and Accountability Plan (LCAP), which accompanies the new funding, 
requires California districts to develop, adopt, and annually update a 3-year accountability plan that 
includes identifying goals and measuring progress for student subgroups across multiple 
performance indicators. The state requires indicators from state assessments (the SBAC tests will 
measure Common Core State Standards, and the Early Assessment Program provides information 
to state universities about college readiness) and other kinds of assessments (e.g., Advanced 
Placement tests, English proficiency scores), as well as information about student persistence, 
graduation, college-going, school climate, and parent input and participation. Districts can add to the 
state measures. 

Allocating Funds Based on Student Needs 

Several other states and districts have developed approaches like California’s. For example, 
Massachusetts adopted a weighted student formula funding system in the 1990s that is credited— 
along with its investments in early childhood education, extensive professional development for 
teachers, and new standards and assessments—with propelling large gains in student achievement in 
the state, especially among previously low-achieving students (see Guryan, 2001). Similar plans have 
been proposed in Ohio (Governor Strickland’s Evidence-Based Model (EBM) school funding 
reform plan proposed in 2009, which also included a teacher compensation system to combat the 
inequitable distribution of teachers [Center for American Progress and the Council of Chief State 
School Officers, 2014]), 8 and in Colorado (legislation proposed in 2013 that added weights for low- 


7 For further details, go to 

http://www.mainelegislature.org/legis/bills/getPDF.asp?paper=SP0439&item=l&snum= 125 

8 For further details, go to https://education.ohio.gov/getattachment/Topics/Finance-and-Funding/State - 
Funding-For-Schools/Financial-Reports/District-Payment-Reports/PASS-Summary-FY10-llv2.pdf.aspx 
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income students and English learners, while creating a teaching and leadership investment, an 
innovation fund, and targeted investments in preschool and full-day kindergarten [Herman, 2013]). 

New Mexico created one of the first weighted student funding formulas in the country in 
1974, which divorced student funding from property tax values and allocated dollars based on a set 
of identified student needs (e.g., poverty, English learner status, special education needs) (Center for 
American Progress and the Council of Chief State School Officers, 2014). 9 Because the base funding 
has fallen behind and some district needs have outpaced the plan, legislators have been considering 
updating the formula. Meanwhile, through its recently approved ESEA waiver, New Mexico requires 
schools to monitor the return on investment for interventions in underperforming schools and shift 
strategies if they are not seeing results. The state conducts annual monitoring of this through the 
budgeting process. It also works to identify and replicate interventions showing strong effectiveness. 

Baltimore, New York City, and San Francisco all finance their schools through a Fair 
Student Funding system whereby each school receives its share of the total through a per-pupil 
formula that allocates a base level of funding for each student and supplements this with weights for 
students with particular learning needs and circumstances (Center for American Progress and the 
Council of Chief State School Officers, 2014). 10 Each allows principals to make key financial 
decisions for their schools, generally in collaboration with a school site council, and creates a school 
report card or other data system to record results that are intended to shape future programmatic 
and budget decisions (Center for American Progress and the Council of Chief State School Officers, 
2014). * 11 

Evaluating School Needs and Outcomes Using Multiple Measures 

As suggested by these examples, evaluating the thoughtful use of resources in terms of the 
students’ needs and the outcomes that the investments produce requires a broad and thoughtful set 
of information. During the 1990s, a number of states included multiple measures in their systems of 
accountability. Most of these systems were displaced by NCLB requirements; however, systems that 
report multiple measures have begun to return with the flexibility waivers under ESEA. 

Perhaps the most comprehensive approach has been developed by the California Office to 
Reform Education (CORE) districts in California, which have built on California’s multiple 
measures system under the LCAP and developed a multi-dimensional system for informing school 
accountability and improvement. These districts (Fresno, Long Beach, Los Angeles, Oakland, San 
Francisco, Santa Ana, and Sanger) joined together and were granted a federal flexibility waiver under 
NCLB, which includes the accountability measures shown below. 


9 For further details, go to http://www.nmlegis.gov/lcs/lfc/lfcdocs/finance facts public school funding 
formula.pdf 

10 For further details for Baltimore, go to 

http://www.baltimorecityschools.org/cms/lib/MD01001351/Centricity/Domain/74/FY14- 

AdoptedBudget-CompleteBook.pdf . For New York, see 

http://schools.nyc.gov/AboutUs/funding/overview/default.htm . For San Francisco, see 
http://reason.org/files/weighted student formula sanfrancisco.pdf 

11 For further details, go to 

http://www.baltimorecityschools.org/cms/lib/MD01001351/Centricity/domain/6625/pdf/20120316- 

FSF101 -FINAL.pdf ; http://reason.org/files/weighted student formula sanfrancisco.pdf 
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Figure 10. CORE accountability structure 


Many of these measures are required by the state LCAP, but others, such as the non- 
cognitive skills associated with social-emotional learning, are locally determined and measured. All of 
these measures are considered individually in informing schools about their progress and supporting 
ongoing improvement efforts. 

Other indicators used in California’s LCAP are also reported in CORE districts, including 
measures of students’ opportunities to learn and parents’ opportunities to be involved in their 
children’s education. These include: 

• the availability of qualified teachers, adequate facilities, and necessary materials; 

• student access to a broad curriculum, including the core subjects (including science and 
technology), the arts, and physical education; 

• student access to college coursework and career pathways; 

• evidence of parent participation and opportunities for input. 

To meet federal requirements for identifying low-performing schools, CORE developed a School 
Quality Improvement Index comprised of weighted measures within three domains: 

• Academic (achievement and growth, graduation rate, and persistence rate in grades 8-10, 
together 60% of the index); 

• Social/Emotional (suspension/expulsion, chronic absenteeism, and noncognitive skills, 
together 20% of the index); and 

• School/District Culture & Climate (stakeholder voice/perceptions of students, staff, and 
parents; special education identification; and English learner entry/exit, together 20% of 
the index) (Center for American Progress and the Council of Chief State School 
Officers, 2014). 12 


12 For further details, go to http://www2.ed.gov/policy/eseaflex/approved- 
requests / corerequestfullredacted.pdf 
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To couple resource allocations with identification of school needs, CORE directs improvement 
resources (formative tasks, student remediation courses, professional development for teachers) 
toward any school that falls below certain thresholds (e.g., a specific pass rate on the 10th grade 
California High School Exit Exam), regardless of the school’s overall rankings. CORE has also 
outlined a resource-enriched School Quality Improvement process that builds professional capacity 
in schools that are identified as priority schools, as well as sharing expertise among all schools in the 
consortium (Center for American Progress and the Council of Chief State School Officers, 2014). 13 

Some other states have also begun to develop multiple-measures approaches to assessing 
school performance in ways that are intended to focus attention on key dimensions of learning and 
to create incentives for attending to important outcomes. Generally speaking, under the terms of 
their ESEA flexibility waivers, these states must identify schools as “priority” or “focus” schools 
based only on their math and ELA test scores and, in some cases, graduation rates. However, many 
have proposed using broader measures to inform schools and the public about progress on other 
areas of learning and performance they care about. Several have indicated a desire to include these 
more centrally in the accountability system. Ideally, such indicators of school performance would be 
directly tied to a process by which critical resources are allocated to address school and student 
needs, as is now the case in California (described earlier). 

As an example of expanded measures for evaluating schools, in 2011, Wisconsin replaced 
the Adequate Yearly Progress system with a multiple-measure accountability index comprised of 
student achievement, student growth, achievement gaps, and an indicator of “On-Track to 
Graduation and Postsecondary Readiness” as measured by graduation rates, attendance rates, and 
ACT participation and performance, as applicable for all students and sub-groups (Center for 
American Progress and the Council of Chief State School Officers, 2014). The system takes into account 
other factors, including test participation, absenteeism, and dropout rates. Wisconsin is considering 
future inclusion of additional measures, such as science proficiency and postsecondary enrollment. 

Oregon’s ESEA waiver redesigned the Oregon Report Card for schools and districts to 
incorporate multiple measures, including academic achievement, academic growth, and—for high 
schools—graduation rates, all displayed by subgroup (Center for American Progress and the Council 
of Chief State School Officers, 2014). 14 While not currently considering school climate data a formal 
part of the accountability system, the Oregon Department of Education recently administered a 
statewide survey of public school teachers and administrators to gather information on how 
educators perceive their teaching and learning conditions and school climate. The 2014 Teaching, 
Empowering, Leading, and Learning (TELL) Oregon Survey hopes to deliver insights that can 
impact evidence-based policymaking as well as state and local decisions that improve student 
outcomes and teacher retention (Center for American Progress and the Council of Chief State 
School Officers, 2014). 15 

Illinois’s school rating system also includes the potential for schools to earn bonus points for 
strong results on a school climate survey. Illinois will also include English language proficiency 
exams in its new accountability system, thereby increasing school accountability for the performance 
of English learners. The state will also include science and ACT exams as a measure of college 
readiness for high school students (Center for American Progress and the Council of Chief State 


13 For further details, go to http://www2.ed.gov/policy/eseaflex/approved- 
requests / corerequestfullredacted.pdf 

14 For further details, go to http://www.ode.state.or.us/news/announcements/announcement.aspx?=9408 

15 For further details, go to 

http: / / www.ode.state.or.us / news / announcements / announcement.aspxPID=9578&TypeID=5 
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School Officers, 2014). 16 

New Mexico’s accountability index includes student achievement and growth, graduation 
rate, attendance, and college and career readiness. The state places extra focus on the growth of the 
lowest performing students by giving schools as much credit for the growth of the bottom quartile 
as for the growth of the top three quartiles. In New Mexico’s ESEA waiver, school ratings include a 
student survey that measures opportunity to learn. The state also offers schools bonus points for 
strong student and parent engagement. 

Oklahoma’s approved accountability system uses parent and community engagement and 
school culture indicators as part of school ratings. Schools can earn bonus points for high scores on 
a school climate survey as well as high parent/community volunteer hours. 

Multiple measures can provide a better accounting of what schools are doing and with what 
results. These broader indicators of school performance may help draw attention to areas of growth 
and need that can direct investments and improvement efforts. Whether educators and policymakers 
take these next steps will influence the extent to which schools actually make progress in better 
educating students. For accounting to be translated into genuine accountability, states and districts 
need processes by which they figure out what schools need and then make the investments of 
resources and expertise that will enable educators to act on this knowledge. 

In addition to providing adequate and equitable resources to schools through the state 
funding system, resource accountability may include efforts to provide wraparound services for 
students who live in low-income communities to ensure early childhood learning, health services, 
and before- and after-school supports that level the playing field. Resource accountability can also 
include specific additional investments for schools found to be struggling. In many cases, these 
initiatives are designed to build professional capacity to teach and support students effectively, as 
described in the next section. 

Professional Capacity and Accountability 

One way in which indicators can be translated into actionable ideas for improvement is by 
combining them with a qualitative analysis of what a school is doing—and how it might improve— 
conducted by experts. Much like the inspectorate process in many other countries, School Quality 
Review processes have evolved in some parts of the United States, combining analysis of data with 
on-site review by expert educators, often accompanied by peer review from inside or outside the 
school. 

Analyzing teaching and school practices to evaluate the extent to which they represent a 
professional standard of instruction and care is a key element of enforcing professional 
accountability for practice. Because of the evidence that School Quality Review processes enhance 
the professional knowledge of practitioners who are involved, we also include them here as a 
component of professional capacity-building. In systems that add ongoing expert support for school 
improvement to the review process, this capacity-building element is even stronger. 

Evaluating and Supporting Professional Practice through School Quality Reviews 

During the 1990s, New York state developed a School Quality Review (SQR) with the 
assistance of David Greene, one of Her Majesty’s Inspectors from Great Britain. The review began 
with a school self-assessment that provided a foundation for a visiting team of educators from other 
schools guided by an expert inspector using protocols that directed attention to the areas of school 
operations to be evaluated, with a strong focus on teaching and learning. The review examined 


16 For further details, go to 

http://www.americanprogress.org/wp-content/uploads/issues/2012/07/pdf/nochildwaivers.pdf 
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student work as well as instruction in classrooms. Similar reviews were developed in Chicago, 
California, and Rhode Island, among other places. Though discontinued at the state level during a 
round of budget cuts, a version of the SQR remained in New York City and evolved over time, and 
continues today. 

The New York Quality Review involves 2- or 3-day school visits by experienced educators 
to each NYC school (Center for American Progress and the Council of Chief State School Officers, 
2014). 17 The external evaluator visits classrooms, speaks with school leaders, and uses a rubric to 
evaluate how well the school is organized to support student achievement. A Quality Review rating 
is then given to each school along with a report that is published on its DOE website. 

Under its ESEA waiver, New York state engages a somewhat different diagnostic process to 
support low-performing schools and districts using a program of Distinguished Educators. These 
highly effective educators are appointed by the commissioner to assist schools and districts whose 
prior intervention efforts have failed. These educators “provide an intensive review of district and 
school systems, structures, operations, and facilities and develop an action plan; assess the district’s 
capacity to promote and support teaching and learning within all schools in the district; work with 
district administration and the board of education to review data, analyze district and school 
structures, plan for improvement, and assist in targeting district priorities; facilitate increased student 
performance across the district; and recommend administrative and operational improvements to 
strengthen systems” (Center for American Progress and the Council of Chief State School Officers, 
2014). 

Kentucky established a Program Review system to assess the quality of programs in Arts & 
Humanities, Writing, and Practical Living and Career Studies (Center for American Progress and the 
Council of Chief State School Officers, 2014). 18 Program Reviews are conducted internally at the 
school level three times a year by staff, parents, students, and relevant community members. An 
annual external review at the district level is then conducted at the end of each school year whereby 
district review teams are able to request and review Program Internal Review reports prepared by 
schools throughout the year. Kentucky also engages in a highly regarded diagnostic review process 
of struggling schools and districts using the AdvancED assistive technology, which has coupled a 
new form of accreditation with follow-up services to support school improvement. 

Ohio conducts School Improvement Diagnostic Reviews (SIDRs) for schools identified as 
underperforming based on test data. 19 SIDRs are conducted by an external team of experienced and 
skilled reviewers who follow a standard protocol for collecting evidence to diagnose a school’s 
strengths and weaknesses. SIDR teams are responsible for making prioritized recommendations that 
are presented to the school several weeks later in a diagnostic report. Like Ohio, as part of their 
ESEA flexibility waivers, a number of states are doing diagnostic review for at least some of their 
schools. These states include Arkansas, Florida, Illinois, Minnesota, New Jersey, Oklahoma, and 
Wisconsin. 

Supporting School Improvement by Sharing Expertise 

As part of its new accountability system, California has created the California Collaborative 
for Educational Excellence (CCEE). The CCEE will mobilize expertise in the state to help districts 


17 For further details, go to 

http://schools.nyc.gov/Accountability/tools/review/Process/Overview/default.htm 

18 For further details, go to http://education.ky.gov/curriculum/pgmrev/Documents/Program Review 
Guide Section 1 .pdf 

19 For further details, go to http://education.ohio.gov/getattachment/Topics/School-Improvement/Race-to- 
the-Top/Resources-and-Reports/RttT-Monthly-Quarterly-Reporting-November-2013.pdf.aspx 
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improve the quality of teaching and school leadership, and meet the needs of special populations 
(English learners, special education students, students at risk of dropping out). It will offer 
particularly intense assistance to districts or schools that are struggling to meet the goals established 
in the Local Control and Accountability Program, but its services will be available to schools and 
districts upon request. The collaborative will sponsor a system of review by expert educators and peers 
that can help build a learning system within the state to stimulate the transfer of knowledge and best 
practices and encourage innovation, experimentation, evaluation, and adaptation. CCEE will not 
only strengthen the state’s capacity to assist schools and districts that need help, but also validate and 
share information about effective practices. 

Pairing highly successful schools with other schools needing support is another means of 
helping schools share expertise, which has been highlighted in studies of Shanghai’s extraordinarily 
successful school system. This strategy has been taken up by the California CORE districts, which 
pair high- and low-performing schools to share best practices, and help teachers at these schools 
work together to learn from each other. Massachusetts and Tennessee also pair high-growth schools 
with low-performing schools to share best practices. 

Supporting Educator Capacity and Accountability 

The heart of a professional accountable system is a set of elements that ensures that 
educators are carefully selected, receive a high-quality preparation that enables them to acquire 
essential knowledge and skills, are licensed based on useful evidence of effectiveness, supported 
through high-quality induction and professional learning opportunities, and make sound personnel 
decisions—including opportunities for advancement that support further sharing of expertise— 
through thoughtful evaluation, supervision, and career ladders. Professionally accountable systems 
also ensure that well-qualified educators are readily available to all students across the state, which 
requires attention to recruitment incentives, including service scholarships, and adequate and 
equitable salaries and working conditions that provide motivation to stay. 

Although the nation as a whole has lost ground on this agenda during recent years of federal 
and state budget cuts, a number of states have taken substantial steps toward creating an integrated 
set of professional supports and requirements. For example, California has long had some of the 
most rigorous standards for entering teacher education in the nation, with nearly all candidates 
preparing at the graduate level, and examinations of academic skills and subject matter knowledge 
required for entry. The state also launched the nation’s first performance assessments of teaching for 
licensure some years ago. California was also the first state to offer a state-funded multiyear 
induction program for beginning teachers. It has recently added administrator performance 
assessments and a required induction program for administrator licensing, as well, while overhauling 
its standards for teacher and administrator preparation. 20 

The new preparation standards require deeper knowledge of how to teach English learners 
and other students with special needs, as well as content pedagogical knowledge that incorporates 
the Common Core State Standards. A new accreditation system will enforce stronger standards and 
attend to program outcomes by collecting and reporting common data across programs—such as 
graduate and employer evaluations of program quality, pass rates on teacher performance 
assessments, and entry and retention rates in teaching—and using these to target programs for 
scrutiny that appear to be struggling. 


20 For more information, see information about the Commission on Teacher Credentialing at 
http://www.ctc.ca.gov/ 
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When California enacted the CCSS, it allocated $1.25 billion for professional development 
for educators, and it is developing a range of curriculum and learning resources to support districts 
in this work. The state is the first in the nation to authorize and fund Peer Assistance and Review 
Programs to strengthen teacher evaluation statewide, and it has a long-standing statute requiring the 
use of teacher observations and student learning evidence in evaluations. It is now supporting 
districts by documenting and disseminating model programs that can share expertise across the state. 

Delaware has recently raised the entry and exit requirements for teacher preparation and 
focused more attention on the clinical preparation candidates receive. Under SB 51, candidates must 
now have a 3.0 GPA or pass an academic skills test to enter teacher education. To exit, they must 
pass a more rigorous test of content knowledge and demonstrate effective teaching through a 
performance assessment (U.S. Department of Education, 2014). Teacher candidates must participate 
in ongoing residency experiences that include working with a cooperating teacher, participating in 
parent/teacher conferences and professional learning communities, and teaching students while 
being observed by their mentors. 

Delaware’s new teachers and administrators receive support and mentoring. Delaware is one 
of only three states that requires and funds multiyear new teacher induction and makes program 
completion a requirement for licensure advancement. It also is one of only five states to require 3 
years of induction support. The state provides funding for mentors for beginning teachers (New 
Teacher Center, n.d.) and for beginning principals. A Delaware Leadership Academy at the 
University of Delaware offers mentoring and professional learning opportunities for principals and 
other school leaders (Darling-Hammond, Meyerson, LaPointe, & Orr, 2009). 

The Delaware Department of Education maintains ongoing professional development 
opportunities for teachers through a set of approved professional development clusters; through 
subject matter networks like the Delaware Reading Project, Writing Project, Science Coalition, 
Technology Partners; and through ongoing professional learning opportunities in areas like 
Response to Intervention and Positive Behavior Supports. It has recently launched an initiative led 
by a group of accomplished teachers across the state to develop materials and supports for job- 
embedded professional development around the Common Core State Standards. 

The state has leveraged its evaluation system to retain effective teachers and principals 
through the Delaware Talent Cooperative, which provides retention awards to highly effective 
teachers and leaders willing to work and stay in high-need schools (U.S. Department of Education, 
2014). It has also leveraged its evaluation system to inform teacher and principal preparation and 
development through a new Evaluation Report System database (Center for American Progress and 
the Council of Chief State School Officers, 2014). 21 

Massachusetts has also worked to create a comprehensive system of supports and 
requirements for educator knowledge and skills, with high standards for entry implemented through 
a series of assessments of academic skills and subject matter for teacher entry and licensing; 
strengthened requirements for program approval; required induction programs offered by trained 
mentors for both beginning teachers and administrators (New Teacher Center, 2011); and recent 
initiatives to implement performance assessments for licensure for both teachers and administrators. 

The state offers incentives for academically able candidates to prepare for teaching through a 
tuition waiver for aspiring teachers already in college who maintain a 3.0 grade point average and 
commit to teaching in a shortage field for 2 years in Commonwealth schools, along with a 
scholarship program, much like the highly successful North Carolina Teaching Fellows program, 
which attracts qualified high school students to the teaching profession by providing 4-year tuition 


21 For further details, go to 

http://www.americanprogress.org/wp-content/uploads/issues/2012/07/pdf/nochildwaivers.pdf 
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and fees scholarships (Massachusetts Department of Higher Education: Office of Student Financial 
Assistance, 2014). 

Once in the profession, all educators maintain an individual professional development plan, 
and the state sponsors and funds a wide range of professional learning opportunities. The 
department offers free professional development institutes for teachers and administrators during 
the summers, focusing on understanding learning standards, promoting quality instructional 
practices, and helping educators develop an understanding of high-quality curriculum within subject 
matter fields. The state also encourages and enables teachers to access learning opportunities from 
universities, districts, and other sources, as well as job-embedded opportunities, such as mentoring, 
peer coaching, taking and offering seminars, or collaborating on new curriculum units, all of which 
can help fulfill recertification requirements and promote ongoing learning (Massachusetts 
Department of Elementary and Secondary Education, 2000). 

Massachusetts’s new teacher evaluation process is tightly tied to these learning opportunities. 
One of the more sophisticated in the nation, it draws on evidence of teaching practice from 
observations, staff, and student feedback; teachers’ professional contributions; and multiple sources 
of evidence about student learning in a judgment system that is tied to goal-setting and professional 
learning (Massachusetts Department of Elementary and Secondary Education, 2014). 

Conclusion 

We offer these ideas about a new paradigm for accountability in the spirit of beginning a 
conversation that might ultimately result in a policy framework with the potential to allow the 
United States to move forward in its aspirations to educate all students for the demands of the world 
they are entering. We recognize that considerable discussion and debate will be needed before a new 
approach can take shape, and that states will differ in the specific approaches that fit their contexts 
and political cultures. 

Nonetheless, we believe it is imperative to get this national discussion started, as the only 
current consensus is that our current system is not adequate to meet the needs of our schools and 
children, especially those in increasingly under-resourced communities. 

We believe that a new conception of accountability can help the nation meet its aspirations 
for preparing college- and career-ready students by: 

• developing assessments that are more focused on 21st century learning skills and used in 
ways that support improvement in teaching and learning; 

• creating stronger, more multidimensional ways of evaluating schools and more 
sophisticated strategies for helping them improve; 

• addressing the opportunity gap that has allowed inequalities in resources to deprive many 
students of needed opportunities to learn; and 

• developing an infrastructure for professional learning and accountability (e.g., higher 
quality preparation, professional learning, evaluation, and career advancement for 
individuals, plus sharing of expertise across schools) that allows educators to acquire and 
share the knowledge and skills they need to enable students to learn. 

The gauge of a new system should be the outcomes it enables. True accountability should allow 
schools to be both responsible for high-quality professional practice and responsive to students’ 
needs within the context of their families and communities. An effective accountability system 
should give students, parents, and governments confidence that schools are focused on what matters 
most and capable of helping each child connect to a productive future. 
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Appendix A 


Accountability Pillar Overall Summary 
Annual Education Results Reports - Oct 2008 
Province: Alberta 


/dlberia 

EDUCATION 



Notes: 

1) Student Learning Achievement: PAT Values reported are weighted averages of PAT Acceptable and PAT Excellence results. Courses included: ELA (Grades 3, 6, 9), Math (Grades 3, 6, 9), Social Studies (Grades 6, 9), Science (Grades 6 only), French Language 
Arts (Grades 6, 9), Frangais (Grades 6, 9). 

2) Student Learning Achievement: Diploma Exam Values reported are averages of Diploma Acceptable and Diploma Excellence results, weighted by the number of students enrolled in each course. 

3) Overall evaluations can only be calculated if both improvement and achievement evaluations are available. 

4) The ACOL measures are not evaluated as they are not part of the Accountability Pillar and are included only to enable inclusion in the AERR and 3-Year Education Plan reports. 

5) Data values have been suppressed where the number of students is less than 6. Suppression is marked with an asterisk (*). 
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